Gabrielle Bartomeo, Binish Chandy, Zach Dravis, Burcu Kaniskan, Niteen Kumar, Betsy Rosalen
March 25, 2018
The survey data included:
Variables: Country, GenderSelect & Age
Demographics Insight:
|
Top 5
|
|
|---|---|
| Country | count |
| United States | 4034 |
| India | 2648 |
| Other | 974 |
| Russia | 570 |
| United Kingdom | 511 |
| Brazil | 461 |
|
Bottom 5
|
|
|---|---|
| Country | count |
| Kenya | 58 |
| Romania | 58 |
| Belarus | 53 |
| Czech Republic | 53 |
| Chile | 51 |
| Norway | 51 |
|
Top 5 Respondents
|
||
|---|---|---|
| Country | GenderSelect | count |
| United States | Male | 3195 |
| India | Male | 2244 |
| United States | Female | 839 |
| Other | Male | 833 |
| Russia | Male | 470 |
| United Kingdom | Male | 424 |
Let’s take a peek at a day in the life of a Data Scientist and try to figure out what a data scientist does.
Key Attributes: TimeGatheringData, TimeModelBuilding, TimeProduction, TimeVisualizing, and TimeFindingInsights.
Key Attributes: 18 Learning Platforms (Arxiv,Blogs,College,Company,Conferences,Friends,Kaggle,Newsletters,Communities,Documentation,Courses,Projects,Podcasts,SO,Textbook,TradeBook,Tutoring,YouTube)
Understanding what working professionals want to learn could give us insight into what skills are most valued in the field.
These questions were asked to all participants.
Mostly Master’s degrees followed by Bachelor’s then doctoral degrees.
Let’s look at the different Machine Learning/Data Science methods.
| Machine Learning/Data Science |
|---|
| Random Forests |
| Deep learning |
| Neural Nets |
| Text Mining |
| Genetic & Evolutionary Algorithms |
| Link Analysis |
| Rule Induction |
| Regression |
| Proprietary Algorithms |
| I don’t plan on learning a new ML/DS method |
| Ensemble Methods (e.g. boosting, bagging) |
| Factor Analysis |
| Social Network Analysis |
| Monte Carlo Methods |
| Time Series Analysis |
| Other |
| Bayesian Methods |
| Survival Analysis |
| MARS |
| Anomaly Detection |
| Cluster Analysis |
| Decision Trees |
| Association Rules |
| Uplift Modeling |
| Support Vector Machines (SVM) |
Distribution of Machine Learning/Data Science methods with formal education.
| Options | Freq |
|---|---|
| Data Visualization | 5022 |
| Logistic Regression | 4291 |
| Cross-Validation | 3868 |
| Decision Trees | 3695 |
| Random Forests | 3454 |
| Time Series Analysis | 3153 |
| Neural Networks | 2811 |
| PCA and Dimensionality Reduction | 2789 |
| kNN and Other Clustering | 2624 |
| Text Analytics | 2405 |
| Ensemble Methods | 2056 |
| Segmentation | 2050 |
| SVMs | 1973 |
| Natural Language Processing | 1949 |
| A/B Testing | 1936 |
| Bayesian Techniques | 1913 |
| Naive Bayes | 1902 |
| Gradient Boosted Machines | 1557 |
| CNNs | 1417 |
| Simulation | 1398 |
| Recommender Systems | 1158 |
| Association Rules | 1146 |
| RNNs | 891 |
| Prescriptive Modeling | 851 |
| Collaborative Filtering | 793 |
| Lift Analysis | 650 |
| Evolutionary Approaches | 436 |
| HMMs | 419 |
| Other | 391 |
| Markov Logic Networks | 255 |
| GANs | 244 |
Data visualization remains the most frequently endorsed data science method.
The same data in tabular format:
| Selections | Freq | RelativeFreq | Degree |
|---|---|---|---|
| Data Visualization | 1236 | 0.0944232 | Bachelor’s Education |
| Logistic Regression | 989 | 0.0755539 | Bachelor’s Education |
| Decision Trees | 847 | 0.0647059 | Bachelor’s Education |
| Data Visualization | 1129 | 0.0756348 | Doctoral Education |
| Cross-Validation | 1046 | 0.0700744 | Doctoral Education |
| Logistic Regression | 1031 | 0.0690695 | Doctoral Education |
| Neural Networks | 24 | 0.0808081 | High School Education |
| Data Visualization | 23 | 0.0774411 | High School Education |
| Text Analytics | 18 | 0.0606061 | High School Education |
| Data Visualization | 2331 | 0.0835873 | Master’s Education |
| Logistic Regression | 2022 | 0.0725069 | Master’s Education |
| Cross-Validation | 1821 | 0.0652992 | Master’s Education |
| Data Visualization | 150 | 0.0897129 | Professional Education |
| Logistic Regression | 121 | 0.0723684 | Professional Education |
| Decision Trees | 119 | 0.0711722 | Professional Education |
| Data Visualization | 137 | 0.1008837 | Some Post Secondary Education |
| Logistic Regression | 97 | 0.0714286 | Some Post Secondary Education |
| Decision Trees | 87 | 0.0640648 | Some Post Secondary Education |
105 variables in 4 likert scale categories
Plus a few basic demographic fields
30% go by “Data Scientist”
20% go by “Scientist/Researcher” or “Software Developer/Software Engineer”
| CurrentJobTitleSelect | total | percent |
|---|---|---|
| Data Scientist | 644 | 30.51 |
| Scientist/Researcher | 225 | 10.66 |
| Software Developer/Software Engineer | 212 | 10.04 |
| Data Analyst | 185 | 8.76 |
| Other | 177 | 8.38 |
| Researcher | 138 | 6.54 |
| Machine Learning Engineer | 102 | 4.83 |
| Engineer | 73 | 3.46 |
| Statistician | 71 | 3.36 |
| NA | 71 | 3.36 |
| Business Analyst | 59 | 2.79 |
| Computer Scientist | 47 | 2.23 |
| Predictive Modeler | 41 | 1.94 |
| Programmer | 21 | 0.99 |
| DBA/Database Engineer | 19 | 0.90 |
| Operations Research Practitioner | 16 | 0.76 |
| Data Miner | 10 | 0.47 |
About 73% are college students
| StudentStatus | total | percent |
|---|---|---|
| Yes | 113 | 73.38 |
| No | 41 | 26.62 |
55% are “focused on learning mostly data science skills” regardless of academic status.
| StudentStatus | LearningDataScience | total | percent |
|---|---|---|---|
| Yes | Yes, I’m focused on learning mostly data science skills | 63 | 55.75 |
| Yes | Yes, but data science is a small part of what I’m focused on learning | 50 | 44.25 |
| No | Yes, I’m focused on learning mostly data science skills | 23 | 56.10 |
| No | Yes, but data science is a small part of what I’m focused on learning | 18 | 43.90 |
Learners - Top 3 ways to learn data science:
Friends are the least useful learning platform.
Data Visualization is at the top of the list.
Only 38.8% of Learners said Visualizations were a “Necessary” job skill to learn!
The most frequently used of the programming languages are R and Python. But do those that use R recommend R or Python? And do those that use Python recommend R or Python? In other words, do those survey takers feel that others should first and foremost study the languages they themselves have taken up, or perhaps with their insight, know to suggest the language of the two they themselves did not learn?
Thus the following questions were explored:
What is the distribution of following programming languages Kaggle survey takers used in the past year:
What is the distribution of programming language recommendations by following programming languages Kaggle survey takers used in the past year:
There are 2 variables used in this section of the analysis :
LanguageRecommendationSelect=(What programming language would you recommend a new data scientist learn first? (Select one option) - Selected Choice)
WorkToolsSelect= For work, which data science/analytics tools, technologies, and languages have you used in the past year? (Select all that apply) - Selected Choice
## [1] 16716 229
## [1] 7955 229
Let’s examine the above graph of LanguageRecommendationSelect
## [1] 7955 5
Finally, true to the word “value,” considerations have to be made regarding pay. The compensation received by survey takers for their work in either R or Python needs quantification to discover which language earns a data scientist more overall and in general.
Three variables were used:
There was also the variable “id” that was created for the purpose of this report, acting as a way to identify each individual survey taker, and the variable “work_tools” which was a derivative of WorkToolsSelect, breaking the lists down into their individual components.
| Minimum | 1st Quartile | Median | Mean | 3rd Quartile | Maximum | Standard Deviation | |
|---|---|---|---|---|---|---|---|
| Python | $0.00 | $53,000.00 | $100,000.00 | $112,826.14 | $145,000.00 | $2,000,000.00 | $122,425.21 |
| R | $0.00 | $58,000.00 | $87,000.00 | $98,177.64 | $130,000.00 | $550,000.00 | $67,487.91 |
On the contentious debate on which Machine Learning/Data Science methods Data Scientists are most excited about learning in the next year as the most valued Data Science Skills
On Data Science Methods Used on the Job
Learners vs. Employed Data Scientists
On Data Science Activities
On the contentious debate on R vs Python as the most valued Data Science Skills